Using RStudio on Hoffman2

Hoffman2 Happy Hour

Charles Peterson

🎉 Welcome to Hoffman2 Happy Hours

🎉 Welcome to the Hoffman2 Happy Hours

  • Short presentations on HPC-related topics and practical uses of Hoffman2
  • Thoughts for future “Happy Hour” topics? 💡

📧 cpeterson@oarc.ucla.edu

📖 Access the Workshop Files

This presentation and accompanying materials are available on UCLA OARC GitHub Repository 🔗 https://github.com/ucla-oarc-hpc

You can view the slides:

Clone the repository for workshop files:

git clone https://github.com/ucla-oarc-hpc/H2HH_rstudio

Clone the H2 Rstudio repository:

git clone https://github.com/ucla-oarc-hpc/H2-RStudio

RStudio Information

🖥️ What is Rstudio

A powerful IDE for R, data visualization, and script management


But why do you want to use RStudio on Hoffman2 when you can use your own computer???

RStudio on Hoffman2 provides access to:

  • Higher Memory
  • multi-core processing
  • GPUs
  • Your Hoffman2-hosted data

🌐 RStudio Formats

There are two main (free) RStudio formats that researchers can use

🖥️ RStudio Desktop

🌐 RStudio Server

🚀 RStudio on Hoffman2

RStudio Desktop

  • Must connect to H2 via X11 forwarding or remote desktop
  • Runs R and Rstudio from H2 modules
  • sluggish interaction depending on connection

RStudio Server

  • R/Rstudio from Docker container
  • Isolate container OS image
    • Separate version of R that was built from Hoffman2 modules
    • **DON’T load R modules for server version

Running Rstudio Desktop

🖥️ Connect via X11

Login to Hoffman2 via X11 Forwarding

  • On your local machine, ensure you have an X11-enabled application installed:
    • Windows: Use MobaXterm, XMing, or Cygwin. For WSL, install an X server like VcXsrv or XMing.
    • macOS: Install and launch XQuartz.
    • Linux: Ensure X11 forwarding is enabled on your terminal.
  • Use the following command to login to Hoffman2 with X11 Forwarding:
ssh -X USER@hoffman2.idre.ucla.edu

The -X flag enables your local computer’s X11 server to display the RStudio GUI.

🔗 Connect via Remote Desktop

Connect via Remote Desktop

  • Alternatively, you can use a remote desktop application:
  • NoMachine or X2Go: Configure the session with the correct server address and login credentials. Set the desktop resolution and environment based on your preferences.

Information on using X11 and Remote Desktop on Hoffman2 can be found on our workshop Visualization application on HPC

📜 Running Rstudio Desktop

Once connected to Hoffman2 via X11 or Remote Desktop,

  • Start an interactive session to request resources:
qrsh -l h_data=10G,h_rt=1:00:00
  • Load the necessary modules:
module gcc/10.2.0
module R/4.2.0
module load Rstudio
  • Launch RStudio:
module load Rstudio
rstudio &

If everything is set up correctly, the RStudio GUI will appear on your local computer or remote desktop.

Running Rstudio Server

🛠️ Starting RStudio

Get An Interactive Job

Containers cannot run on login nodes.

  • You MUST use a compute node


qrsh -l h_data=10G

Modify the qrsh to meet your RStudio computing needs

  • More memory and/or job time
qrsh -l h_data=50G,h_rt=5:00:00
  • More cores
qrsh -l h_data=10G -pe shared 10
  • Using GPUs
qrsh -l h_data=10G,gpu,V100

🛠️ Setting up directoreis

Create Temp Directories

  • Rstudio requires writable temporary directories outside of the container

  • It can be anywhere you have write access

  • Create directories in $SCRATCH/rstudiotmp



mkdir -pv $SCRATCH/rstudiotmp/{var/lib,var/run,tmp}

🛠️ Loading RStudio

Load the Apptainer Module

  • Apptainer is software that will run the Rstudio container


module load apptainer

RStudio Server on Hoffman2 created from Docker

  • Separate R from modules on Hoffman2
    • DO NOT load R modules
    • R packages may need to be reinstalled

🛠️ Starting RStudio



export RSTUDIO_VERSION=4.1.0
apptainer run \
      -B $SCRATCH/rstudiotmp/var/lib:/var/lib/rstudio-server \
      -B $SCRATCH/rstudiotmp/var/run:/var/run/rstudio-server \
      -B $SCRATCH/rstudiotmp/tmp:/tmp \
         $H2_CONTAINER_LOC/h2-rstudio_${RSTUDIO_VERSION}.sif
  • apptainer run
    • Starts the RStudio container
  • -B $SCRATCH/rstudiotmp/[dir]:[/dir]
    • Mounts tmp directories to the container
  • $H2_CONTAINER_LOC/h2-rstudio_4.1.0.sif
    • Location of RStudio container
    • Can be change to different RStudio versions
  • Information will display about RStudio session
    • Note the compute node name and port number.
    • Displays ssh -N -L ... info to be ran
    • You will see a Rstudio Password
      • Needed to open Rstudio

Note

KEEP THIS TERMINAL OPEN UNTIL YOU JOB IS DONE

🛠️ Connecting to Rstudio

Once Rstudio process is started:

  • Open another terminal on your local computer

  • Run the port forward command

    • Creates a connection from local computer to compute node
  • Change port 8787 if needed
  • nXXX is the compute node name
  • username is your Hoffman2 username
ssh  -N -L 8787:nXXX:8787 username@hoffman2.idre.ucla.edu 

This will create a remote ssh connection from your local computer to the Hoffman2 compute node running Rstudio via port 8787

🛠️ Opening RStudio

  • Finally, open a web browser
    • Type URL of RStudio Server
    • Will ALWAYS be localhost
    • Change port 8787 if needed




http://localhost:8787

Running Rstudio - The Easy Way

  • h2_rstudio.sh
    • Script that runs everything from the previous slide
    • Starts Rstudio and opens a web browser for you
    • Runs on your local computer (not Hoffman2)

h2-studio.sh Information

Look at our Github page

  • Download script
wget https://raw.githubusercontent.com/ucla-oarc-hpc/H2-RStudio/main/h2_rstudio.sh
chmod +x h2_rstudio.sh
  • To display how to use this script
./h2_rstudio.sh -h
  • Run script
    • Replace username with Hoffman2 username
./h2_rstudio.sh -u username

Tested Platforms

Mac’s terminal app

Window’s WSL2

MoboXterm

GitBash

📝 RStudio Script

This RStudio Script is currently on our GitHub page

This script will create an RStudio session on a compute node on Hoffman2.

REQUIRED OPTIONS:
  -u [username]    Your Hoffman2 username (mandatory)

OPTIONAL PARAMETERS:
  -m [MEMORY]     Memory requirements in GB (default: 10 GB)
  -t [TIME]       Time of RStudio job in HH:MM:SS (default: 2:00:00)
  -v [VERSION]    RStudio version (default: 4.1.0)
  -p              Request high-priority queue (highp)
  -g [GPUTYPE]    Request GPU resources, where GPUTYPE can be 'V100', 'A100', A6000, etc.

HELP:
  -h              Show this usage message

📦 Info on RStudio Container

  • Rstudio container was built using Docker
    • Based on RStudio images from the Rocker Project
    • Hoffman2 containers located at $H2_CONTAINER_LOC
    • RStudio containers are named:
      • h2-rstudio_X.Y.Z.sif
      • Where X.Y.Z is the R version
  • View all available RStudio containers by running
module load apptainer
ls $H2_CONTAINER_LOC/h2-rstudio*sif

📍 User Packages

  • Separate build of R and
  • R packages installed in unique directory
    • ~/R/APPTAINER/h2-rstudio_4.1.0 (for h2_rstudio-4.1.0.sif)
  • HPC Container files
    • Docker and definition files for Hoffman2 containers
    • RStudio Dockerfiles have all you need to build RStudio

R Package Installs

  • Some R packages require extra libraries or software in the container
  • Contact us to update this container
    • OR you can modify the Dockerfile for your own container

🛠️ Error fixing

  • If Rstudio does not at start up
    • Possibly due to previous RStudio not shutdown correctly
  • Clear out any tmp directories files
rm -rf $SCRATCH/rstudiotmp 
mkdir -pv $SCRATCH/rstudiotmp/{var/lib,var/run,tmp}
  • Clear out RStudio config files
rm -rf $HOME/.config/rstudio
rm -rf $HOME/.local/share/rstudio

🖥️ Terminal Access

  • Access to a Hoffman2 terminal in RStudio

📊 Using Batch R

  • Instead of interactive RStudio, you can run R as a non-interactive batch job
    • Use R from inside RStudio container as a qsub job
  • Create job script
#!/bin/bash
#$ -cwd
#$ -o rstudio_batch.out.$JOB_ID
#$ -j y
#$ -l h_rt=3:00:00,h_data=10G
#$ -pe shared 1

# Load the apptainer module
. /u/local/Modules/default/init/modules.sh
module load apptainer

# R Version
export RSTUDIO_VER=4.4.0

# Use the RStudio container to run R code
apptainer run $H2_CONTAINER_LOC/h2-rstudio_${RSTUDIO_VER}.sif R CMD BATCH myRtest.R
  • Then run this job script
qsub rstudio_batch.job

📝 Summary

  • RStudio Desktop on Hoffman2
    • Connect via X11/Remote Desktop
    • Runs R from H2 modules
  • RStudio Server on Hoffman2
    • Access through on your web browser
    • Runs R from inside containers

🙏 Thanks and Happy Computing!

Questions? Comments?